Quality Assurance Engineer III

Department Overview

Duties

• Own the end-to-end test strategy, covering front-end applications, backend services, APIs, data pipelines, and AI-enabled features. 
• Run functional, integration, regression, and end-to-end testing that follows each business workflow from source data to the user-facing report. 
• Build and maintain UI automation with Playwright (TypeScript / JavaScript) and API and data tests with pytest, integrated into GitHub Actions as quality gates. 
• Test batch and streaming pipelines on BigQuery, Dataflow, Pub/Sub, Confluent Kafka, Apache Airflow (Astronomer), and Talend, including backfills, re-runs, idempotency, and failure recovery. 
• Validate data across raw, bronze, silver, and gold layers with SQL reconciliation, checking completeness, freshness, duplicates, referential integrity, and schema drift. 
• Build dbt tests (generic, singular, unit, and contract), and automate data quality rules and a DQ scorecard with Dataplex Data Quality. 
• Verify KPI calculations, business rules, aggregations, and filters in the web app, Tableau, and MicroStrategy reports against source data, including consistent satisfaction calculations across reporting surfaces. 
• Assure Talend-to-dbt modernization and pipeline refactoring with parallel runs, row and hash checks, validation reports, and side-by-side output comparisons. 
• As AI-powered features are introduced, test them on Vertex AI and Gemini, including retrieval data quality, bias, drift, prompt regression, and LLM / RAG output accuracy, grounding, and hallucination. 
• Test performance, load, and security, including response times, BigQuery query cost, PAM-based role access and application role provisioning, row and column-level security, PII masking, and GDPR delete and notify flows across user groups and markets. 
• Act as level-2 support behind the run-operations team, troubleshooting issues across UI, APIs, and data with Datadog, PagerDuty, and Cloud Monitoring and Logging, leading root cause analysis, and maintaining troubleshooting guides. 
• Support the annual release cycle and new market onboarding through regression, deployment and post-release validation, configuration and access checks, localization and translation testing, and UAT. 
• Use AI tools to generate test cases, test data, and automation scripts, and review every AI-generated test for correctness and coverage. 

Qualifications

• 7–11 years of professional experience in software and data quality engineering, with 5+ years building test automation and 2+ years on GCP. 
• Strong SQL, especially BigQuery, and Python (pytest, pandas). 
• Hands-on UI and API automation with Playwright (TypeScript / JavaScript) and Postman. 
• Hands-on dbt testing and orchestration with Apache Airflow (Astronomer). 
• Good grasp of data modeling, medallion architecture, and warehouse concepts such as slowly changing dimensions, fact and dimension tables, and CDC. 
• Experience with Dataplex data quality and dbt tests. 
• Experience with CI/CD and Git, especially GitHub Actions. 
• Shift-left mindset that automates first, defines clear quality gates, and explains data defects in terms of business impact. 
• Responsible use of AI tools in testing, including keeping confidential and personal data out of prompts. 
• Background in Computer Science, Engineering, or a related field with a bachelor's degree or higher. 
• Preferred: Java test automation (TestNG or JUnit 5, REST Assured); testing approaches for ML and GenAI models; LLM evaluation frameworks (Ragas, DeepEval, or Vertex AI Gen AI Evaluation); performance testing with JMeter or Gatling; Talend job validation; data governance, lineage (Dataplex), and privacy rules such as GDPR; GCP Professional Data Engineer or Professional ML Engineer certification; and retail or quick service restaurant data. 
• Good to know: Selenium or Cypress; Great Expectations, Soda, or Monte Carlo; Vertex AI pipelines, Feature Store, or Model Monitoring; and BDD with Cucumber.